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1.  INTRODUCTION 


The  pressure  alticude  information  reported  by  the  aircraft  plays  an 
important  role  in  the  enhancement  of  safety  and  efficiency  of  flight 
in  the  National  Airspace  System.  The  altitude  information  serves  as 
vital  input  for  a  number  of  today's  and  future  ground-based 
automation  functions  including  surveillance,  vertical  separation 
assurance*  conflict  alert,  and  minimum  safe  altitude  system  (MSAW). 
The  accuracy  of  the  reported  altitude  depends  heavily  on  the 
aircraft  altimetry  system.  The  altimeter  senses  the  ambient 
pressure,  converts  the  pressure  into  electrical  signal,  quantizes 
the  signal  into  discrete  level,  and  sends  it  to  the  transponder 
(Mode  C)  for  digital  transmission.  Currently,  the  Mode  C  data  are 
reported  at  100  ft  increments. 

With  the  planned  implementation  of  the  Mode  S  ground  stations  and  in 
particular,  the  capability  of  the  Mode  S  transponders  to  report 
altitude  at  a  higher  resolution  (25  ft  increments),  there  have  been 
suggestions  from  both  the  international  and  U.S.  aviation 
communities  to  examine  the  potential  benefits  of  reduced 
quantization  for  altitude  data  [1,2].  For  the  ground-based  Air 
Traffic  Concrol  (ATC)  system,  the  primary  interest  lies  in  the 
design  of  an  effective  altitude  tracker  aimed  at  improving  aircraft 
surveillance  and  other  safety-related  automation  functions. 

In  the  airborne  segment,  the  FAA  has  developed  a  family  of  Traffic 
Alert  and  Collision  Avoidance  Systems  (TCAS)  as  backup  to  the  ground 
ATC  system.  In  particular,  the  TCAS  II  system  provides  escape 
maneuver  advisories  automatically  in  the  vertical  plane.  This 
system  cracks  the  altitude  and  altitude  rate  of  Che  intruder  and  own 
TCAS-equipped  aircraft  to  determine  Che  hazard  of  mid  air 
collision.  The  tracking  of  Che  intruder's  alticude  is  based  on  data 
quantized  to  100  ft  (Mode  C)  or  in  the  future,  an  option  for  25  ft 
(Mode  S)  increments. 

For  own  TCAS  altitude  tracking,  the  input  data  can  be  either  at 
100  ft  increments  or  finely  quantized.  Since  TCAS  II  is  expected  to 
be  installed  in  air  carriers,  the  onboard  altimeter  is  generally  of 
high  precision,  including  an  air  data  computer  (ADC).  Depending  on 
che  ADC  type,  digital  altitude  data  of  1  ft  increment  or  analog 
synchro  outputs  are  available.  The  synchro 


outputs  can  ba  quantized  to  small  increments  such  as  under  10  £t  £or  TCAS 
applicat ion. 

Efforts  are  also  underway  by  the  FAA  to  collect  altimetry  data  with  the 
objectives  of  quantifying  altimetry  errors  at  high  and  low  altitude  regimes. 
The  purpose  of  the  data  collection  program  for  high  altitude  is  to  evaluate 
the  feasibility  of  reduced  vertical  separation  standard  above  flight  level 
290.  On  the  other  hand,  the  objective  of  the  altimetry  data  collection 
program  at  low  altitude  is  to  determine  the  distribution  of  the  altimetry 
error  for  the  general  aviation  aircraft  population.  Both  of  these  two  data 
collection  programs  involve  the  analysis  of  reported  altitude  (Mode  C)  error 
relative  to  the  true  altitude.  Highly  accurate  ground  based  systems  are  used 
to  measure  the  true  aircraft  altitude. 

Common  to  the  aforementioned  efforts  of  altimetry  data  collections  and  the 
design  of  the  altitude  trackers  is  the  need  to  characterize  the  altitude 
error  statistically  at  the  output  of  the  quantizer.  This  error  is  defined 
herein  as  the  difference  between  the  altitude  ac  Che  output  of  the  quantizer 
and  the  aircraft's  true  altitude.  Thus,  this  error  consists  of  the 
quantization  error  plus  the  error  prior  to  quantization.  The  latter  error 
includes  the  error  associated  with  the  onboard  altimetry  system  (before 
quantization)  and  the  deviation  from  Che  standard  atmospheric  model  for 
pressure-co-alt itude  conversion. 

In  calculating  the  mean  and  the  standard  deviation  of  the  altitude  error  at 
the  output  of  the  quantizer,  frequently  it  has  been  assumed  that  Che  standard 
deviation  of  the  alcimecry  error  (<rn)  before  quantization  is  comparable  or 
greater  chan  Che  quantization  level  (Q).  This  assumption,  coupled  with  Che 
additional  assumption  chat  the  input  error  is  Gaussian,  leads  to  the 
well-known  result  chat  the  output  quantization  error  is  uniformly  distributed 
within  the  quantization  bin.  However,  the  standard  deviation  of  Che  altimetry 
error  before  quantization  can  be  quite  small  (such  as  a  few  feet)  relative  to 
che  quantization  level;  the  consequence  is  that  Che  uniform  distribution 
assumption  may  not  be  valid  except  for  small  Q. 

Another  common  assumption  is  that  the  mean  of  Che  error  prior  to  quantization 
is  zero.  This  assumption  is  not  always  satisfied  since  the  static  system  of 
che  altimeter  or  che  use  of  che  standard  atmospheric  model  for 
pressure-co-altitude  conversion  can  introduce  a  constant  error  or  bias. 

This  report  presents  a  mathematical  anaLysis  of  che  mean  and  standard 
deviation  of  che  altitude  error  ac  che  output  of  the  quantizer.  The  formulae 
derived  are  very  general  in  chat  they  do  not  assume  an  uniform  distribution 
for  Che  quantization  error.  Furthermore,  che  results  are  exact  and  applicable 
to  all  values  of  c rn  and  Q.  The  only  assumption  made  is  chat  che  distribution 
ac  che  input  of  che  quantizer  is  Gaussian.  Numerical  results  are  also 
presented  for  Q*130  ft,  25  ft,  and  5.25  ft,  and  rn  ranging  from  2.5  ft  to 
25  ft. 


The  next  section  discusses  the  error  model  used  in  Che  analysis  and  che 
previous  work  in  chis  area*  Section  3  derives  che  theoretical  results  £or  che 
mean  and  scandard  deviation  of  che  output  altitude  error.  Section  4  provides 
the  numerical  results,  as  well  as  a  comparison  with  che  uniform  distribution 
model.  The  last  section  summarizes  the  results  of  the  analysis. 


2.  ERROR  MODEL  AND  PREVIOUS  WORK 


Figure  1  depicts  the  model  used  in  Che  analysis  of  Che  altitude  error  at  t.ie 
output  of  the  quantizer.  The  aircraft's  true  altitude  as  a  function  of  time 
is  represented  by  h(t),  which  may  include  aircraft  sinusoidal  motion.  The 
true  altitude  is  corrupted  by  an  additive  term,  n(t),  representing  the 
altimetry  error  and  other  sources  of  error  prior  to  quantization.  This  error 
includes  those  attributable  to  static  source  error  and  transducer  error.  The 
degraded  altitude  information  x(t)  is  quantized  at  a  level  Q  to  produce  the 
output  y(c).  The  quantized  altitude,  y(t),  can  be  thought  of  as  composed  of 
the  crue  altitude  and  an  error  term,  e(t).  It  is  the  determination  of  the 
mean  and  standard  deviation  of  the  output  error  term  e(c)  that  is  of  primary 
interest . 

Ic  is  noced  chat  che  analysis  of  Che  impact  of  e(t)  on  altitude  tracking  is 
outside  the  scope  of  this  report'  since  ic  depends  on  the  specific  tracker  of 
interesc.  However,  the  results  in  characterizing  Che  first-order  statistics 
of  e(t)  are  applicable  to  the  analysis  of  nonlinear  and  linear  cracking 
algorithms. 

The  additive  error  term,  n(t),  is  modeled  as  a  Gaussian  process  of  arbitrary 
mean,  b(c),  and  standard  deviation  <rnCt>.  These  two  parameters  can  be  a 
function  of  time  to  account  for  their  time  variation  as  a  function  of  altitude 
and  atmospheric  condition.  For  economy  of  notation,  b(c)  and  t)  are 
denoted  as  b  and  respect ively. 

The  parameter  b  represents  the  constant  altimetry  error  or  bias  error  before 
quantization,  whereas  crn  denotes  Che  random  component  of  Che  error  or  jitter. 
For  air  carrier  type  of  aircraft  such  as  the  L-1011,  the  <rn  before 
quantization  is  on  Che  order  of  several  feet  [3].  Preliminary  analysis  of 
actual  flight  data  from  B-747  and  Airbus  300  also  shows  the  same  order  of 
magnitude  for  <rn  . 

Since  n(t)  is  a  Gaussian  random  process,  x(t),  a  linear  function  of  n(c),  is 
also  a  Gaussian  process  with  che  following  mean  and  variance: 


x(t)*E[x(t)l  (1) 

*E(  h(  t)+n(  c )  )*h(  t  )4-b 

where  £  denotes  the  mathematical  expectation  operator 


The  analysis  of  che  output  error  e(t)  is  generally  quite  complicated  because 
of  che  nonlinear  nature  of  che  quantizer.  The  altitude  quantizer  under 
consideration  is  a  round-off  type,  as  shown  in  Figure  2.  For  simplicity,  it 
is  assumed  that  quantization  occurrs  at  the  pre-selected  thresholds  with 
negligible  error.  Since  che  quantizer  performs  a  nonlinear  c rans format  Lon  of 
inpuc  x(c),  che  output  altitude  y(t)  is  generally  non-Gauss ian . 


Quantizer 

y(t)  y 

to) _ 

measured  alcitude 
y(t)»h(t)+e(t) 


h(t):  true  aircraft  altitude 
n(t):  altimetry  error  before 
quantization;  Gaussian 
distribution  with  mean  b 
and  variance 

x(t):  corrupted  altitude 
input  to  quantizer 
y(t):  output  altitude 
e(t):  altitude  error  at  quantizer 
output;  defined  as  y(t)-h(t) 


Figure  1-  Error  Model 


Roundoff  Quantizer  (Q» 
—No  Quantization 


The  statistical  effect  of  quantization  has  been  widely  studied  in  many 
fields.  Quantization  error  is  unavoidable  in  digital  systems  when 
analog-to-digital  conversion  is  required.  Perhaps  the  earliest  work  in 
establishing  the  relationship  between  the  moments  at  the  output  of  the 
quantizer  and  those  at  the  input  is  known  as  Sheppard's  corrections  for  group 
data  [4].  The  pioneering  work  in  the  rigorous  analysis  of  the  quantization 
effect  was  performed  by  Widrow  [5],  who  first  established  the  statistical 
distribution  at  the  output  of  the  quantizer  and  the  necessary  conditions  for 
the  output  to  be  uniform  and  white.  His  results  are  also  known  as  the 
quantization  theorem.  Extensions  of  his  work,  as  well  as  previous  efforts, 
include  those  of  references  {6,7,3]. 

The  following  summarizes  the  previous  key  results  relevant  to  the 
determination  of  the  mean  and  variance  of  the  quantized  output  error,  e(c), 
assuming  the  input  to  the  quantizer,  x(t),  is  Gaussian  : 

i)  For  "large"  ^n/Q  ratio  (e.g.,  <J"n/Q  *1)  »  it  is  well-known  that  the 

quantization  error,  defined  by  y( c)-x(c) ,  is  uniformly  distributed  over 
the  quantization  interval  and  chat  the  input  x(t)  and  Che  quantization 
error  are  uncorrelated.  This  leads  to  the  folowing  relationship  between 
che  variance  of  che  quantized  altitude  error  Co  Che  variance  aC  the 
input : 

2  2  2 

°e  " °n  +  -2—  (V 

12 

In  che  above  equation,  Q2/12  represents  che  quant izac ion  error  due  co 
the  uniform  distribution  model. 


ii)  For  <rn/Q  *  1/3  the  following  approximation  has  been  derived  { 4  ,5  ] 


For  x(c)^0. 


e(  c  )■£( e(  t ) ]*  -  £  exp  ( -2  t2  /Q 2  )  sin  f  Z  tt  x(  c  )  A 
tt  V  0  1 


For  x(t)*0. 


2*n 


XI  2 

exp  ( -2  tt-  cr  /Q  ) 


iii)  For  all  range  of  ^n/Q  and  x(t)»0,  the  variance  of  e(t)  is  given 

by  1 8 1  : 

lit  1  ^  i  22^2 

cr  -<rQ  +  +  4o-q  \  (-1)  exp  (-2  *  i  <rn  / q  ) 

12  £i 

+  ^  l^exp  (-2  n2  tZaaZ /Q2  ) 


It  can  be  noced  in  (ii)  above  for  moderace  crn/q  ratio  and  for  nonzero  mean 
x(c)  case,  che  variance  of  e(t)  has  not  been  derived.  Similarly,  in  (iii)  Che 
mean  and  the  variance  of  the  error  at  the  quantizer  output  are  not  available 
when  the  mean  of  x(t)  is  not  zero.  For  ATC  applications,  the  mean  of  x(c)  is 
only  zero  when  both  the  aircraft  altitude  and  the  bias  of  the  altimetry  error 
are  zero. 


3.  DERIVATION  OF  THE  OUTPUT  ALTITUDE  ERROR 


The  derivation  of  che  mean  and  the  variance  of  the  altitude  error  at  the 
output  of  the  quantizer  is  relatively  straight-forward,  albeit  algebraically 
cumbersome.  The  altitude  error  at  the  output  of  the  quantizer  is  defined  as 
the  difference  between  the  output  altitude  and  the  true  aircraft  altitude. 

That  is,  the  output  error  e(c)  is, 

e(t)-y(t)  -  h(t)  ',6) 

The  output  error  e£t)  defined  above  is  related  to  che  commonly  known 
quantization  error  by  an  additive  noise  term  n(t).  This  can  be  seen  by  noting 
that  che  quantization  noise  is  defined  as  : 

e  (t)*y(t)  -  x(t) 

^  *y(t)  -  h(c)  -  n(t) 


or, 


y(c)»e^U)  +  h(c)  +  n(t) 


(7) 


Substituting  the  above  into  equation  (6),  the  output  error  e(t)  is  related  to 
che  quantization  error  e^Ct)  by: 


eCc)-e^(c)  +■  n(  t )  (8) 

In  general,  eq(t)  and  n[c)  are  correlated,  i.e., 

E[eq(c)n(t)]  *  E[eq(t)l*E[n(t)l .  The  correlation  between  the  two  error  terms 
on  the  right-hand  side  of  equation  [8)  decreases  as  /Q  increases  [81 


The  mean  and  che  variance  of  e(c)  are: 


Ele(t)l-S[y(c)]  -  h(c) 
e( t)»y( t)  -  h( c) 


[9) 


z 

^•Variance  Iy(t)  -  h(t)) 

•Variance  [y(t)l 

-Ety^Cc)  1  -  [  7U)]2  :  10) 

It  can  be  observed  from  equations  9,10  chat  the  mean  and  the  variance  of  the 
alcitude  error  at  the  output  of  the  quantizer  are  related  directly  to  the 
first  and  second  moment  of  the  quantized  output  yCt).  To  calculate  the 
moments,  the  technique  of  the  characteristic  function  is  adopced. 

Denoting  y(t)  and  x(t)  at  a  particular  time  t  as  Y  and  X  respectively,  the  kth 
moment  of  Y  is  related  to  its  characteristic  function  <$^(p)  by: 


3-1 


Only  the  first  two  moments  are  derived,  i.e.,  k-l,2.  The  relationship 

between  the  characteristic  function  <Sy(  p)  and  9g(p),  where  $x(p)  is  the 
characteristic  function  of  X,  is  given  by  (5,7]: 


co  ^ 

<SY(p)-  y  #  (  p  -  2  IT  i/Q)  •  sin  ((  p-2  tt  i/Q) ♦  Q/2  ] 
.  ^  x  L  (p-2  TT  i/Q) .  Q/2 


i««to 


(12) 


The  characteristic  function  of  Che  Gaussian  random  variable  X  with  mean  X  and 


variance  <r_  is  given  by 


1  Z 


$x(p)»exp  ( jXp  -  <rn  p  /2) 


(13) 


Performing  Che  differentiation  in  accordance  with  equation  (11)  and  making  use 
of  equation  (12)  and  (13),  the  first  two  moments  of  the  output  altitude  error 
have  been  derived.  The  details  of  the  derivation  are  provided  in  Appendix  A. 
From  the  appendix,  the  first  and  second  moment  of  Y  are  given  by: 


E(Y)-X  +■  £  V  ( ~l)isin  (2tt  iX/Q)  exp  /Q1) 


1.2  1 


TT  1  1 

1-  1 


:n) 


2  Z  _Z  2  1  A  i  -  7  12  1 

E(Y  )-  o-n  +  ( X)  +  +  Q  V  (-1)  cos( 2  tt  iX/Q)  exp  ( -2  tt  i  on  /Q  ) 

12  ~  L.  ~T2_ 


i-1 


«  i  /  ^  ,  _  ,  _  _  'l  2  Z  2  -y 

+  C-l)  (»Tri<r„  COS  ^2_n_iX  j+  2X  sin  ^  2  tt  iXUexp  (-2tt  i  <rn  /Q  ) 

•I  _  .  ^  TlC 


i-1 


(15) 


Substituting  the  mean  of  X  from  equation  (1)  into  the  above  two  equations  and 
using  equations  (9,10),  the  formulae  for  the  mean  and  the  variance  of  che 
altitude  error  at  the  output  of  the  quantizer  can  be  shown  to  be: 

Ele(c)]«e 

-E(Y)  -  h 

-b  +■  Q  ^  ( -1)^*  exp  (-2*t  i  <r5  /Q1)  sin  (2  tt  iC  h-t-b ) /Q  ]  (  16) 


<re  -E  { Y  )  -  (  Y  ) 

2  1  i  i 


2  1  x  “  i  \  X  X  X 

'  Tn  +  2l  +  2_  \  (~p  exp  C -2  -rr  i  crQ  /Q  )  cos  {2  it  i(h+b)/Q) 
12  ir1  ^  ix 

l  XT'  1  i  1>  i  2 

+  +  >  (“D  exp  ("2ir  i  ^  /Q  )  sos  {2  it  i(h+b)/Q] 

1-1  2 
_  f  S_  ^  (-1)1  exp  (-2irai*<r*  /Q*')  sin  [2  tt  iCh+b)/Q)  \ 
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For  ^n/Q  *  1/3 ,  equations  (16,17)  can  be  approximated  by  the  first  term  of 
the  infinite  series.  Hence, 

For  <rn/Qii/3  , 

—  2  Z  2. 

e*b  "  jL.  exp  (-2  tt  <rn/Q  )  sin  1 2  it  (h-l-b)/Q] 
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4.  NUMERICAL  EVALUATION 


Ic  can  be  seen  from  equations  (16)  and  (17)  that  the  mean  and  the  variance  of 
the  altitude  error  at  the  output  of  the  quantizer  are  a  function  of  four 
variables:  h  (aircraft's  true  altitude)  ,  b  (altimetry  bias) ,  <rn  and  Q. 
Furthermore,  even  if  the  input  bias  b  and  do  not  vary  with  time,  the  output 
error  parameters  of  e  and  re  can  be  time  dependent  since  the  altitude  of  the 
aircraft  generally  changes  with  time  unless  it  is  flying  level. 

To  reduce  the  number  of  variables  for  graphical  display  purposes,  the 
following  modified  altitude  error  is  defined: 


■  (-1)  exp  ( —2 tt^ i  cr^/Q5')  [2  tt  i(h+b)/Q]  (18) 

it  2-,  i 
1*1 

Tlje  above  simply  subtracts  the  bias  from  Che  mean  output  altitude  error.  The 
e  and  (Te  are  function  of  three  variables  in  ( h+b) /Q ,a^and  Q.  Since  the 
variation  of  e^  and  oj  is  sinusoidal  with  respect  to  (h+b)/Q,  ic  is  only 
sensitive  to  the  fractional  remainder  of  (tH-b)/Q,  or  R[(h+b)/Q].  The  function 
R  is  defined  as, 

R(z)  *  z  -  Inceger  (z) 

where  Inceger  (z)  denotes  Che  integer  part  of  z. 

(e.g.,  Inceger( 10 .5)*10 ;  Integer( -1.5)*-1) 

Therefore,  0  4  |.R(z)J  ^  1.  It  can  be  noted  chat  two  aircraft  at  different 
altitudes  can  result  in  the  same  fractional  remainders  and  therefore  the  same 
output  mean  and  variance  if  Che  bias  (b)  and  Q  are  che  same.  For  example,  if 
hf+-b*20,025  ft  and  Q,*100  ft,  and  h^4-b*50,025  ft  and  Q^lOO  ft,  Chen 

R(  2002 5/ 100) *1/4 

R(  50025/ 100)-l/4 

A  computer  program  has-been  written  to  compute  the  output  altitude  error 
parameters,  1*  and  <re  ,  based  on  equations  17  and  18.  The  specific  Q  levels 
evaluated  are  Q*100,  25  ,  6.25  ft.  The  input  altimetry  error  <r  was  varied 
from  2.5  ft  to  25  ft. 

The  plotting  of  the  output  altitude  error  vs.  Rl(h+-b)/Q]  »Tn»  and  Q  are  given 
in  two  forms.  The  first  form  presents  the  errors  vs.  Rl(hfb)/Q]  wichtipand  Q 
as  parameters  in  order  to  heighten  the  sensicivity  of  errors  with  respect  to 
altitude  level  of  the  aircraft  and  Q.  The  second  form  emphasizes  Che 
variation  of  error  with  respect  to  <rn  ,  with  R((h+b)/Q]  and  Q  as  plotting 
parameters.  These  two  forms  are  somewhat  redundant  in  information  data  base, 
but  wich  different  emphasis. 


Without  loss  of  generality,  only  the  positive  values  of  R(h+b/Q)  will  be 
illustrated  in  the  figures  herein.  That  is,  the  emphasis  is  on  the  important 
case  in  which  the  the  aircraft's  true  altitude  plus  bias  is  higher  than  zero 
feet. 


4.1  Mean  Altitude  Error  at  the  Output  of  Quantizer 


The  mean  altitude  error  at  the  output  of  the  quantizer,  e  ,  as  a  function  of 
the  fractional  remainder,  R[Ch+b/Q],  is  shown  in  Fig.  (3)  for  Q*100  ft  and  in 
Fig.  (4)  for  Q“25  ft.  It  can  be  observed  from  these  two  figures  that  the  mean 
output  altitude  error  "e  is  an  odd  function  of  R[Ch+b)/Q]  about  Rl'h+b)/Q]  * 
1/2.  It  is  also  noted  that  1*  is  zero  at  R[(h+b)/Q]*0 ,1/2 ,  regardless  of 
the  quantization  levels. 


Another  observation  is  that,  for  the  same  input  >  the  peak  e  decreases 
with  reduced  Q.  For  Of,  *2.5,  the  peak  "e*"  values  for  Q*100  ft  and  Q*25  ft  are 
approximately  43  ft  and  7.3  ft  respectively. 


It  can  also  be  noted  that ,  as  3J1  increases ,  the  peak  e  decreases 


regardless  of  Q.  In  fact,  from  Figure  (3),  as  d^/Q  increases  to 

i  n/*  ►  ^  ^  a  si'musai^  us 


l/i 


corresponding  to  0^*25  ft ,  "e  approaches  a  sinusoid  with  respect  to 
R[(h+b)/Q) .  This  agrees  well  with  the  known  result  that  for  moderate  values 
of  0^/Q,  only  the  first  term  of  the  infinite  series  in  equation  (Id)  needs 
to  be  used.  As  <J*/Q  becomes  very  large,  e  approaches  zero. 


It  has  often  been  mentioned  that  it  is  the  that  has  the  most  impact  on 
altitude  rate  estimation,  rather  than  e*"  .  This  is  true  when  the  mean 
error  does  not  fluctuate  significantly  with  time.  For  high  Q  case  such 

as  Q»i00ft,  and  for  some  unique  combination  of  aircraft  altitude  profile 
and  (7^,  the  contribution  of  '3*'  to  rate  error  estimation  can  be 
appreciable.  This  aspect  needs  further  investigation. 


The  variation  of  e  with  input  altitude  error  is  shown  on  Fig.  5  and  5 
for  Q*100  ft  and  Q»25  ft  respectively.  It  can  be  observed  from  Fig.  6  for 
Q*25  ft  that  for  %  *10  ft,  the  output  ”5^  is  zero.  However,  for  Q*100  ft,  the 
peak  e*  varies  between  27  ft  to  -27  ft  for  the  same  range  of 


4.2  Standard  Deviation  of  Altitude  Error  at  The  Output  of  the  Quantizer 


The  standard  deviation  of  the  output  altitude  error  as  a  function  of 
R((h+b)/Ql  for  Q*100  ft  is  shown  in  Fig.  7.  It  can  be  observed  that  is  an 
even  function  of  R[(h+b)/Q]  about  R[ (h+b)/Q]»l/2.  The  peak  Og  is  50  ft, 
occuring  at  R(  (h+b) /Q ]*l/2 .  That  is,  if  b*0  ,  the  maximum  ^  would  occur  at  the 
round-off  threshold  point.  This  agrees  with  the  intuition  that  at  low  Cr/Q, 
any  small  jitter  about  the  round-off  threshold  point  would  push  the  quantized 
altitude  rip  or  down)  to  the  next  quantized  altitude,  resulting  in  of 
approximately  Q/2.  As  will  be  noted  later,  thi3  observation  is  not  true  for 
high  3fi/Q  case. 


Mean 

Output  Sid 

Altitude 

Error 


It  can  be  noted  from  this  figure  that  Cg  decreases  as  R[(h+b)/Q]  moves  away 
from  R[(h+b)/Q]M/2  and  reaches  a  minimum  at  R[(h+b)/Q]*0.  Furthermore,  for 
the  smallest  noise  case  of  <£*2.5  ft,  the  spread  of  eg  about  R[(h+b)/Q]-l/2  is 
very  small.  As  3^  increases,  flg  becomes  more  dispersed. 

Figure  8  displays  the  variation  of  for  Q*25  ft.  In  comparison  with  the  QMOO 
ft  and  * 7.5  ft  cases,  the  same  observations  about  also  applies  to  Q*25 

ft.  However,  for  MO  ft,  is  constant  and  does  not  depend  on  R[  (h+b) /Q ] . 

Figures  9,  10,  and  11  display  the  standard  deviation  of  the  quantized  error  as 
a  function  of  the  input  altitude  error  (  )  for  QM00  ft,  25  ft,  and  6.25  ft 

respectively.  The  dashed  lines  in  these  figures  represent  the  ideal  case  of  Q 
approaching  zero  or  no  quantization  case. 


For  a  specific  Cg  ,  it  can  be  seen  from  these  figures  that  as  Q  decreases,  the 
spread  of  the  <J£  decreases  and  the  maximum  3g  also  decreases.  It  can  be  noted 
that  as  Q  drops  to  6.25  ft,  ag  almost  approaches  that  for  the  non-quantization 
case.  Furthermore,  Cg  is  almost  independent  of  the  aircraft  altitude. 

Noise  suppression  or  reduction  can  be  observed  for  Q*100  ft  and  Q*25  ft  (Fig. 

9  610)  for  specific  input  altimetry  error  (  Jn  ,  b) ,  quantization  level 
(Q),  and  true  aircraft  altitude  (h).  This  means  that  the  output  is  less 
than  the  input  Og  because  of  altitude  quantization.  In  general,  this 
phenomenon  occurs  at  low  % /Q  levels  and  a  subset  of  R[(h+b)]/Q  values.  For 
example,  for  QMOO  ft  and  input  ^  *10  ft.  Fig.  (9)  shows  that  the  output  x 
is  less  than  the  input  for  R[  (h+b) /Q  ]»0  ,1/8 ,1/4  ,3/6  ,7/8 . 

Another  interesting  example  is  the  case  in  which  QMOO  ft,  b*0 ,  and  the  true 
aircraft  altitude  is  exactly  divisible  by  100  (coinciding  with  the  flight 
level  notation).  In  this  case,  R(h/Q)*Q,  Fig.  (9)  shows  that  the  output  Cg  is 
the  lowest,  as  compared  to  other  R(h/Q)  values.  Furthermore,  the  mean  output 
error  (ej^is  also  the  lowest  (see  Fig.  5).  However,  when  the  last  two  digits 
of  the  aircraft  altitude  are  50,  the  output  Cg,  is  the  largest  while  the  mean 
output  error  is  the  smallest. 

In  general,  an  aircraft  in  flight  will  experience  the  full  spectrum  of 
variation  as  its  altitude  changes  with  time. 


4.3  Comparison  with  the  Uniform  Altitude  Error  Model 

For  ease  of  analysis,  it  has  frequently  been  assumed  that  the  error  (variance) 
at  the  output  of  a  quantizer  device  can  be  calculated  by  summing  the  input 
error  and  the  quantization  error  using  the  RSS  method  (see  equation  2).  The 
variance  of  the  quantization  error  is  assumed  to  be  Q4/ 12  based  on  the  uniform 
distribution  of  this  error.  However,  the  uniform  distribution  model  is  not 
universally  valid,  especially  in  low  5i/Q  cases. 


Input  0^  ( ft) 


Figure  10~  Output  0^  Vs.  Input  0^  for  Q*25 


The  accuracy  of  this  model  is  compared  with  the  results  given  in  Figures  9, 

10,  11.  It  can  be  seen  from  these  figures  that  the  <re  values  predicted  by  the 
uniform  distribution  model  fall  in  between  the  maximum  and  the  minimum  of  the 
true  theoretical  values.  As  Q  decreases,  the  uniform  model  becomes  more 
accurate  for  the  range  of  input  noise  examined.  As  Q  approaches  6.25  ft, 
negligible  difference  exists  between  the  computed  (exact)  values  and  those 
based  on  the  uniform  distribution  model. 

For  quantization  levels  of  100  ft  and  25  ft,  the  validity  of  the  uniform 
distribution  model  depends  on  the  <r„  of  interest.  For  example,  for  Q*25  ft, 
it  can  be  observed  from  Fig.  10  that  when  &n  *10  ft  approximately,  there  is 
little  difference  between  the  values  provided  by  the  uniform  distribution 
model  and  the  exact  calculation. 

The  appropriateness  of  the  uniform  distribution  model  is  probably  more 
dependent  on  the  application  at  hand.  If  approximate  analysis  is  the  desired 
goal,  then  the  uniform  distribution  model  would  suffice. 


5.  CONCLUSIONS 


The  analysis  and  numerical  results  provided  herein  show  that  reduced  altitude 
quantization  has  a  positive  impact  in  reducing  the  errors  at  the  output  o£  the 
quantizer.  Specific  conclusions  reached  include  the  following: 

-The  mean  and  the  variance  of  the  altitude  error  at  the  quantizer  output 
vary  with  the  fractional  remainder  of  (h+b)/Q,  the  input  altitude  error 
variance,  and  the  quantization  level  Q.  Since  an  aircraft  in  flight  generally 
has  a  dynamic  altitude  profile,  the  quantized  altitude  error  also  changes  with 
time . 


-Reducing  the  quantization  level  also  reduces  the  peak  mean  output 
altitude  error.  Reducing  Q  from  100  ft  to  25  ft  diminishes  the  peak  mean 
altitude  error  at  the  quantizer  output  from  43  ft  to  7.3  ft.  for  the  range  of 
input  error  statistics  considered. 

-As  quantization  level  is  reduced,  the  peak  of  the  <re  and  the  spread 
of  <re(max  and  min  of  <re )  are  also  reduced.  Figures  9,10,11  contain  the 
numerical  results  for  various  Q  and  input  statistics.  As  altitude  is  finely 
quantized  such  as  to  6.25  ft,  there  is  negligible  difference  between  the 
quantized  altitude  error  and  the  altitude  error  without  quantization. 

-A  comparison  of  che  approximate  uniform  distribution  model  with  the 
exact  formulae  shows  that  the  former  method  produces  a  value  cre  in  between  the 
maximum  and  the  minimum  of  che  true  value.  For  a  specific  standard  deviation 
of  input  altitude  error,  as  Q  decreases,  the  uniform  distribution  model 
becomes  more  accurate. 
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APPENDIX 


First  and  Second  Moment  of  Quantizer  Output 


The  first  and  second  moment  of  the  quantizer  output  are  derived  in  this 
appendix  assuming  the  input  distribution  is  Gaussian.  The  input  and  the 
output  of  the  quantizer  of  level  Q  are  denoted  by  X  and  Y  respectively.  A 
round-off  type  of  quantizer  with  input  and  output  characteristic  shown  in  Fig. 
[ 2 )  is  assumed. 


The  method  of  characteristic  function  is  used  to  derive  the  first  two  moments 
of  Y.  The  kth  moment  of  the  random  variable  Y  is  given  by: 
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where  ®(p)  is  the  characteristic  function  of  the  quantizer 
^output  Y 


Denoting  the  characteristic  function  of 
the  the  input  and  output  characteristic 
series  J7] : 


X  as  ®xlp),  che  relationship  between 
functions  is  given  by  Che  following 
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The  expected  value  of  Y,  or  first  moment,  is 
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Different iat ing 


$  (p)  with  respect  to  p,  we 


get , 


Carrying  out  Che  above  operations  and  making  use  of  equation  (A-3),  the  first 
moment  of  Y  is  given  by: 

a  t 

E(Y)«  X  -  Q  V  C-i)  <g^(~2  tt  i/Q) 
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From  equation  (A-l),  the  second  moment  of  Y  is. 
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where  Q* and  $"  denote  the  first  and  second 
derivatives  respectively 
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Afcer  considerable  algebraic  manipulations,  Che  second  moment  is  given  by. 
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Ic  should  be  noted  chat  the  above  expression  is  valid  regariless  of  the 
distribution  of  the  input  X.  That  is,  the  normality  assumption  for  the  input 
to  the  quantizer  has  not  been  used. 


Substituting  Che  characteristic  function  of  a  Gaussian  distribution  with  mean 
X  and  variance  r  into  equations  tA-4,  A-5),  the  first  two  moments  of  Y  can  be 
shown  to  be: 
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